Convergence Rates for Greedy Kaczmarz Algorithms, and Randomized Kaczmarz Rules Using the Orthogonality Graph
نویسندگان
چکیده
The Kaczmarz method is an iterative algorithm for solving systems of linear equalities and inequalities, that iteratively projects onto these constraints. Recently, Strohmer and Vershynin [J. Fourier Anal. Appl., 15(2):262-278, 2009] gave a non-asymptotic convergence rate analysis for this algorithm, spurring numerous extensions and generalizations of the Kaczmarz method. Rather than the randomized selection rule analyzed in that work, in this paper we instead discuss greedy and approximate greedy selection rules. We show that in some applications the computational costs of greedy and random selection are comparable, and that in many cases greedy selection rules give faster convergence rates than random selection rules. Further, we give the first multi-step analysis of Kaczmarz methods for a particular greedy rule, and propose a provably-faster randomized selection rule for matrices with many pairwise-orthogonal rows.
منابع مشابه
Convergence Rates for Greedy Kaczmarz Algorithms, and Faster Randomized Kaczmarz Rules Using the Orthogonality Graph
The Kaczmarz method is an iterative algorithm for solving systems of linear equalities and inequalities, that iteratively projects onto these constraints. Recently, Strohmer and Vershynin [J. Fourier Anal. Appl., 15(2):262-278, 2009] gave a non-asymptotic convergence rate analysis for this algorithm, spurring numerous extensions and generalizations of the Kaczmarz method. Rather than the random...
متن کاملConvergence Rates for Greedy Kaczmarz Algorithms
We discuss greedy and approximate greedy selection rules within Kaczmarz algorithms for solving linear systems. We show that in some applications the costs of greedy and randomized rules are similar, and that greedy selection gives faster convergence rates. Further, we give a multi-step analysis of a particular greedy rule showing it can be much faster when many rows are orthogonal.
متن کاملAccelerated Kaczmarz Algorithms using History Information
The Kaczmarz algorithm is a well known iterative method for solving overdetermined linear systems. Its randomized version yields provably exponential convergence in expectation. In this paper, we propose two new methods to speed up the randomized Kaczmarz algorithm by utilizing the past estimates in the iterations. The first one utilize the past estimates to get a preconditioner. The second one...
متن کاملFinding a Maximum Weight Sequence with Dependency Constraints
In this essay, we consider the following problem: We are given a graph and a weight associated with each vertex, and we want to choose a sequence of vertices that maximizes the sum of the weights, subject to some constraints arising from dependencies between vertices. We consider several versions of this problem with different constraints. These problems have applications in finding the converg...
متن کاملLinear convergence of the Randomized Sparse Kaczmarz Method
The randomized version of the Kaczmarz method for the solution of linear systems is known to converge linearly in expectation. In this work we extend this result and show that the recently proposed Randomized Sparse Kaczmarz method for recovery of sparse solutions, as well as many variants, also converges linearly in expectation. The result is achieved in the framework of split feasibility prob...
متن کامل